{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/OpenEnv_gpt_oss_(20B)_Reinforcement_Learning_2048_Game.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# <img width=\"35\" height=\"35\" alt=\"image\" src=\"https://github.com/user-attachments/assets/2700a971-e5d6-4036-b03f-2f89c9791609\" /> OpenEnv: Agentic Execution Environments\n",
        "We're using the new [OpenEnv](https://github.com/meta-pytorch/OpenEnv) library which has over 2000+ environments for RL!\n",
        "\n",
        "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n",
        "<div class=\"align-center\">\n",
        "<a href=\"https://unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png\" width=\"115\"></a>\n",
        "<a href=\"https://discord.gg/unsloth\"><img src=\"https://github.com/unslothai/unsloth/raw/main/images/Discord button.png\" width=\"145\"></a>\n",
        "<a href=\"https://docs.unsloth.ai/\"><img src=\"https://github.com/unslothai/unsloth/blob/main/images/documentation%20green%20button.png?raw=true\" width=\"125\"></a></a> Join Discord if you need help + ⭐ <i>Star us on <a href=\"https://github.com/unslothai/unsloth\">Github</a> </i> ⭐\n",
        "</div>\n",
        "\n",
        "To install Unsloth your local device, follow [our guide](https://docs.unsloth.ai/get-started/install-and-update)."
      ],
      "metadata": {
        "id": "GtkTGCuh6QZy"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hzPgFeIkZn9q"
      },
      "source": [
        "# Goal: Make gpt-oss play games with Reinforcement Learning\n",
        "\n",
        "Our goal is to make OpenAI's open model gpt-oss 20b play the 2048 game with reinforcement learning. We want the model to devise a strategy to play 2048, and we will run this strategy until we win or lose.\n",
        "\n",
        "<img src=\"https://upload.wikimedia.org/wikipedia/commons/thumb/f/f9/2048_win.png/500px-2048_win.png\" height=300 />"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31KIMLJLnHET"
      },
      "source": [
        "# Installation\n",
        "We'll be using [Unsloth](https://github.com/unslothai/unsloth) to do RL on GPT-OSS 20B, and [OpenEnv](https://github.com/meta-pytorch/OpenEnv) for the environment interactions. Unsloth saves 70% VRAM usage and makes reinforcement learning 2 to 6x faster!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "CGoDZwcunHEU"
      },
      "outputs": [],
      "source": [
        "%%capture\n",
        "import os, importlib.util\n",
        "!pip install --upgrade -qqq uv\n",
        "if importlib.util.find_spec(\"torch\") is None or \"COLAB_\" in \"\".join(os.environ.keys()):\n",
        "    try: import numpy; get_numpy = f\"numpy=={numpy.__version__}\"\n",
        "    except: get_numpy = \"numpy\"\n",
        "    !uv pip install -qqq \\\n",
        "        \"torch>=2.8.0\" \"triton>=3.4.0\" {get_numpy} torchvision bitsandbytes \"transformers==4.56.2\" trackio \\\n",
        "        \"unsloth_zoo[base] @ git+https://github.com/unslothai/unsloth-zoo\" \\\n",
        "        \"unsloth[base] @ git+https://github.com/unslothai/unsloth\" \\\n",
        "        git+https://github.com/triton-lang/triton.git@05b2c186c1b6c9a08375389d5efe9cb4c401c075#subdirectory=python/triton_kernels\n",
        "elif importlib.util.find_spec(\"unsloth\") is None:\n",
        "    !uv pip install -qqq unsloth trackio\n",
        "!uv pip install --upgrade --no-deps transformers==4.56.2 tokenizers trl==0.22.2 unsloth unsloth_zoo"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will then install [OpenEnv](https://github.com/meta-pytorch/OpenEnv) from source:"
      ],
      "metadata": {
        "id": "OjifwNNZ7bMx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%%capture\n",
        "!pip install -qqq fastapi uvicorn requests open_spiel\n",
        "!git clone https://github.com/meta-pytorch/OpenEnv.git > /dev/null 2>&1\n",
        "%cd OpenEnv\n",
        "import subprocess, sys, os\n",
        "from pathlib import Path\n",
        "sys.path.insert(0, './src')\n",
        "working_directory = str(Path.cwd().parent.absolute() / \"OpenEnv\")"
      ],
      "metadata": {
        "id": "1yzMMSLR7dNj"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CcLYwLyQLADE"
      },
      "source": [
        "We'll load GPT-OSS 20B and set some parameters:\n",
        "* `max_seq_length = 768` The maximum context length of the model. Increasing it will use more memory.\n",
        "* `lora_rank = 4` The larger this number, the smarter the RL process, but the slower and more memory usage`load_in_16bit` will be faster but will need a 64GB GPU or more (MI300)\n",
        "* `offload_embedding = True` New Unsloth optimization which moves the embedding to CPU RAM, reducing VRAM by 1GB."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 528,
          "referenced_widgets": [
            "83cdc795bf81400787457cb72ccd265c",
            "bee5c215c01349a893277200730f9a2a",
            "39b720a342264a3ca74ec101010671bb",
            "88a111b1d64d4f2688c65ed28a06c62d",
            "c0bf03dbb0bc4e808268fbf8d458fed3",
            "90b4a7e0316d4e2fb9e976ec0c61eb77",
            "b4248f48607144cd835327991f85d0c5",
            "337698676cef42fd8b9d356ca9023be4",
            "a61a310b048e48c4a6434d7e6225f630",
            "140b6a185bfa433fb48a1c878857b216",
            "a654b9b92cd54916bc9ab45fce6fb77a",
            "a16edcdf8b6a4339a5f1c6fe14f35cb1",
            "88001b08f87a4c5aad2fae41346d93ce",
            "8f5a4260c16441f489c4326ed498bed9",
            "191bfb34de6c4e66b0f4debbd137aa61",
            "e7043a72b1be479ea8956228f4a8774c",
            "082e8f0f8492483a94c8168528950efe",
            "b0ae96c9075b441f83eaa019a58c3b30",
            "209256542eb6497689ee291cc3180c01",
            "1b66d8f404854c8da03702634d301a02",
            "1a5f521f88474cd4b7942aea0f8fa888",
            "16c140e98542415fb6b07f8a892185b2",
            "f18bbbacbcd54c9abab52072c3f73c2d",
            "872cbc648fd1484793b3b763a6a47244",
            "364c26a8fda442d1b8b0a752934963ce",
            "ad78fc1270924bb8a94e02b42cf8ff7c",
            "a90ba7ea60434c66b23261547497dce3",
            "eb9940d6e1d047e49c6f3536c8f3ee30",
            "0e77b625c6a2448580275e7009be21c0",
            "3b4e60df969940af9baf165996544914",
            "b3ed99825d3043d097caa08756471d7f",
            "d88119cb71894de6b38cbfd0abd736db",
            "32d3ca1e8e26404ea2eff4c6ae746591",
            "33364a388b964f44bf5503476758ff84",
            "bed86bfe5355486d97cb20137de2901e",
            "919f622c4b594e2499103632a8bbe93f",
            "2f9d7a4281124747b4982683cbf5749e",
            "dfa095324480404db4b2afa2fa2eadd7",
            "1a8ea075a0c7495ca65a6e0c7a04b6d1",
            "9f33729d16874aa2b7e38743e0f96c83",
            "2b14aeaa07cb4d8995b24c6692bd6797",
            "0ed8aae20d164d10883a0a90cff22132",
            "f7d5617bf290407aa5c8eec6306ba853",
            "af9946bbd3dc4120af0cff7eb23b8795",
            "234811e9134c472b9174e680fed95b2d",
            "da4b0b04334348ba967c2cfce3c5a887",
            "27d8e1dfa108416a978dcc40ddf42b4c",
            "edf7bb666eb04b8ea54b2c4dc50a023c",
            "4bf87a423c194fc099efc2d78cd6c3c7",
            "b821c746b5d348eda7b3c27219d9a20d",
            "c7c6a67636eb486bb23f5224e5c9200a",
            "b5162183b39446468e33a2cdca30f758",
            "23b62212dece410dbe2a85c00196da9c",
            "ea1299a844ed42e082cea091c8ce195f",
            "4115eb4d5dd44bae83056a4e63dd3b10",
            "74cb90588496488d9714f10cb77af3b0",
            "0f15dc2ad6404f0d828696a1d150be9e",
            "deb3ae63d3fa486bb9208dd772e4b941",
            "dd9d7e48e0e5412daa6a307f063f6727",
            "808bb5cc236744638b91201e7e09aa84",
            "693372c9491d4e149984f5725bacdd9e",
            "2866c066ebad4f8e954c40b23031fc62",
            "45df16423e9548dab1793573ff0f4884",
            "3b1cd9915698483ebaf87d4635beaa4d",
            "ed121ee923374c44bb486db6e5fbf2c9",
            "bf2f622bb9cd4c4ea70d33ace46163ed",
            "8296f821dad64002a4dea006b2c6b19c",
            "aa0344691dce4ab5a174eb3870953e7c",
            "d49597a1e02244819b4061fa87346ca8",
            "e9a3bd725fd547fcbf5eacefc65a975f",
            "187ef593f27a428da305efa910b0e191",
            "56150486ad3e4004af0ff41742c94e04",
            "57f83c3b71ab4a88880a869537678997",
            "159849a3cfb74fd7a80977bcb9879dd8",
            "50acc2e861954abc9554c690cae30b06",
            "05f69e6251514ea48eb1693ec3a78430",
            "98bfb758b6a147fc93ecf48eed9e0791",
            "cb5d3ce91d18489295ce40bfeb647f26",
            "8e74d760ddab4155b63e56068f4a719f",
            "1947bfabde534fa987e79215cf49cc00",
            "4ca0f5ce85864c629d08fe03b8e7656b",
            "6e158395111247d392eca45f7bc43a6d",
            "15613a5f627e4e94b1cc6b5f60e89130",
            "632709a7f02641faacf76a980c9f6d86",
            "f5244758d8164b71bb2ca15b506f96c3",
            "a904dcaddbfb4eb3b99acc1ad560e388",
            "044eca01c0314107b266903d68920287",
            "be10d0853b08422f8d0eb5b835a29901",
            "bc07f92151a94144bd172d8b0cee312b",
            "68c7876595f54be79239b9e9d5cc060a",
            "4ef28720654642a6930f1626f3614d7d",
            "d5019b1026624b2db6222dc6c7675445",
            "6388045a56d4442ca1ab4b5d3223e864",
            "1bb6b53d018847e28ceec92967305253",
            "8a8b670769844150a9347b036bf614c3",
            "bd8db8bcc0924f07822c37550b0f1e1e",
            "c649e647eb534c1281b57b8e75012f95",
            "13707c1ee0e24d84962157afb6de134f",
            "1ea56ef7280a4854beefd515b1ec0212",
            "d31bc0b1c75b42d384442a966291cafb",
            "61392f2309ac462e858c8510935c708f",
            "30daf6694d5643688031af64657a5b21",
            "46ec6fd59dbf4546af573356153f06f0",
            "014ea07f613b46e5a52887e017e297a0",
            "14d34b3c3d52493985c3d65246b55009",
            "aeee2eeaa46b4f0d9dbec6eb1ba8dcbd",
            "c2d0d41cbf894458b91380f3fd5c187e",
            "0c2f712d7f6b443199c54976d225094f",
            "94cc5d592a164b838185609f5d82064e",
            "958c3e89aa0142559abd0bdd0991e44e",
            "1e4542c5982a4fa2b6bfbee2787ba728",
            "0ace86978f524af3be4a5d3795a70b66",
            "9cfdc7fe2c3d453897eef1e5fcabf530",
            "1e063f1d7ef1420c8d54e73798b46246",
            "caa6354355ac4bd8b87106b853a15e1e",
            "f1f8a6738ba04c6da3c401c83a17b3bd",
            "deb19c6d4c414457aa55c72b2f142828",
            "f7d2c370a02e446aa3eefc5b16b11808",
            "c43418c254e247058506b1df18d64f78",
            "c4c16513140e4132bc5252fb9e7cb6ed",
            "da20cad56c664e2595c1f79a35a3bf6b"
          ]
        },
        "id": "DkIvEkIIkEyB",
        "outputId": "4f238134-dd16-4869-fec4-d67d8a3edc3e"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "WARNING:torchao:Skipping import of cpp extensions due to incompatible torch version 2.8.0+cu126 for torchao version 0.14.1             Please see https://github.com/pytorch/ao/issues/2919 for more info\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "🦥 Unsloth Zoo will now patch everything to make training faster!\n",
            "==((====))==  Unsloth 2025.10.10: Fast Gpt_Oss patching. Transformers: 4.56.2.\n",
            "   \\\\   /|    NVIDIA A100-SXM4-80GB. Num GPUs = 1. Max memory: 79.318 GB. Platform: Linux.\n",
            "O^O/ \\_/ \\    Torch: 2.8.0+cu126. CUDA: 8.0. CUDA Toolkit: 12.6. Triton: 3.4.0\n",
            "\\        /    Bfloat16 = TRUE. FA [Xformers = None. FA2 = False]\n",
            " \"-____-\"     Free license: http://github.com/unslothai/unsloth\n",
            "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model.safetensors.index.json: 0.00B [00:00, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "83cdc795bf81400787457cb72ccd265c"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model-00001-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a16edcdf8b6a4339a5f1c6fe14f35cb1"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model-00002-of-00004.safetensors:   0%|          | 0.00/4.00G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f18bbbacbcd54c9abab52072c3f73c2d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model-00003-of-00004.safetensors:   0%|          | 0.00/3.37G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "33364a388b964f44bf5503476758ff84"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "model-00004-of-00004.safetensors:   0%|          | 0.00/1.16G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "234811e9134c472b9174e680fed95b2d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/4 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "74cb90588496488d9714f10cb77af3b0"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/165 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "8296f821dad64002a4dea006b2c6b19c"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json: 0.00B [00:00, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cb5d3ce91d18489295ce40bfeb647f26"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/27.9M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "bc07f92151a94144bd172d8b0cee312b"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "special_tokens_map.json:   0%|          | 0.00/446 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d31bc0b1c75b42d384442a966291cafb"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "chat_template.jinja: 0.00B [00:00, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1e4542c5982a4fa2b6bfbee2787ba728"
            }
          },
          "metadata": {}
        }
      ],
      "source": [
        "from unsloth import FastLanguageModel\n",
        "import torch\n",
        "max_seq_length = 768 # Can increase for longer RL output\n",
        "lora_rank = 4        # Larger rank = smarter, but slower\n",
        "model, tokenizer = FastLanguageModel.from_pretrained(\n",
        "    model_name = \"unsloth/gpt-oss-20b\",\n",
        "    load_in_4bit = True,\n",
        "    max_seq_length = max_seq_length,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TfeUs-lQJDSq"
      },
      "source": [
        "To do efficient RL, we will use [LoRA](https://arxiv.org/abs/2106.09685), which allows us to only add 1 to 5% of extra weights to the model for finetuning purposes. This allows us to save memory usage by over 60%, and yet it retains good accuracy. Read Unsloth's [GPT-OSS RL Guide](https://docs.unsloth.ai/new/gpt-oss-reinforcement-learning) for more details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8rGa-o3HJCo1",
        "outputId": "d256e0ce-9cfb-4bd4-fc46-a098b0dae8f9"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Unsloth: Making `model.base_model.model.model` require gradients\n"
          ]
        }
      ],
      "source": [
        "model = FastLanguageModel.get_peft_model(\n",
        "    model,\n",
        "    r = lora_rank, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n",
        "    target_modules = [\n",
        "        \"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
        "        \"gate_proj\", \"up_proj\", \"down_proj\",\n",
        "    ],\n",
        "    lora_alpha = lora_rank*2, # *2 speeds up training\n",
        "    use_gradient_checkpointing = \"unsloth\", # Reduces memory usage\n",
        "    random_state = 3407,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N0QnO9_YJBOI"
      },
      "source": [
        "# 2048 game environment with OpenEnv\n",
        "\n",
        "We first launch an OpenEnv process and import it! This will allows us to see how the 2048 implementation looks like!"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from envs.openspiel_env import OpenSpielEnv\n",
        "from envs.openspiel_env.models import OpenSpielAction, OpenSpielObservation"
      ],
      "metadata": {
        "id": "hQrG81XV87-7"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll be using Unsloth's OpenEnv implementation and wrapping the `launch_openenv` with some setup arguments:"
      ],
      "metadata": {
        "id": "-WT0Zu0IcN_r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "global port\n",
        "global openenv_process\n",
        "port = 9000\n",
        "openenv_process = None\n",
        "server = \"envs.openspiel_env.server.app:app\"\n",
        "environment = {\n",
        "    **os.environ,\n",
        "    \"PYTHONPATH\": f\"{working_directory}/src\",\n",
        "    \"OPENSPIEL_GAME\": \"2048\",\n",
        "    \"OPENSPIEL_AGENT_PLAYER\": \"0\",\n",
        "    \"OPENSPIEL_OPPONENT_POLICY\": \"random\",\n",
        "}\n",
        "\n",
        "# Augment Unsloth's OpenEnv creation function\n",
        "import functools\n",
        "from unsloth import is_port_open, launch_openenv\n",
        "launch_openenv = functools.partial(\n",
        "    launch_openenv,\n",
        "    working_directory = working_directory,\n",
        "    server = server,\n",
        "    environment = environment,\n",
        "    openenv_class = OpenSpielEnv,\n",
        ")"
      ],
      "metadata": {
        "id": "zBIye94T8djG"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's see how the current 2048 game state looks like:"
      ],
      "metadata": {
        "id": "P3rMiKLl9Ro2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "port, openenv_process = launch_openenv(port, openenv_process)\n",
        "result = openenv_process.reset()\n",
        "current_state = result.observation\n",
        "current_state"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uBPx0Hho9Xi1",
        "outputId": "bff932f7-a50f-408e-c489-cc39553e4a60"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Unsloth: Creating new OpenEnv process at port = 12724....................................................................................\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "OpenSpielObservation(done=False, reward=None, metadata={}, info_state=[0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 4.0, 0.0, 0.0, 0.0, 0.0], legal_actions=[0, 1, 2, 3], game_phase='initial', current_player_id=0, opponent_last_action=None)"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "First let's convert the state into a list of list of numbers!"
      ],
      "metadata": {
        "id": "4Qz1tRVTAii8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "def convert_to_board(current_state):\n",
        "    n = len(current_state.info_state)\n",
        "    size = int(np.sqrt(n))\n",
        "    board = np.array_split(np.array(current_state.info_state, dtype = int), size)\n",
        "    board = [x.tolist() for x in board]\n",
        "    return board, size\n",
        "convert_to_board(current_state)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7PVeoKW2AmKr",
        "outputId": "83920694-b6ea-4ff2-9ffc-16e461933b46"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "([[0, 0, 0, 0], [2, 0, 0, 0], [0, 0, 0, 4], [0, 0, 0, 0]], 4)"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We also want to pretty print the game board!"
      ],
      "metadata": {
        "id": "hCS56yu29dsG"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "cellView": "form",
        "id": "D9CI4jtgL5mw"
      },
      "outputs": [],
      "source": [
        "#@title (Collapsible) 2048 Game Renderer\n",
        "def render_board(obs, colors: bool = True, border: bool = True, dot_for_zero: bool = True) -> str:\n",
        "    \"\"\"\n",
        "    Pretty-print the board with colors that scale from 0 up to self.target.\n",
        "    Uses ANSI 256-color codes (works in most terminals). Set colors=False to disable.\n",
        "    \"\"\"\n",
        "    import math\n",
        "    b, size = convert_to_board(obs)\n",
        "    mx = max((max(row) for row in b), default=0)\n",
        "    cell_w = max(3, len(str(mx)))\n",
        "\n",
        "    RESET = \"\\x1b[0m\"\n",
        "\n",
        "    # A smooth-ish gradient from cool → warm\n",
        "    # (blue/cyan/green → yellow/orange/red). Tweak or expand as you like.\n",
        "    GRAD = [33, 39, 45, 51, 50, 49, 48, 47, 46, 82, 118, 154, 190, 226, 220, 214, 208, 202, 196]\n",
        "    ZERO_FG = 239  # dim gray\n",
        "\n",
        "    def color_code(v: int) -> str:\n",
        "        if not colors:\n",
        "            return \"\"\n",
        "        if v == 0:\n",
        "            return f\"\\x1b[38;5;{ZERO_FG}m\"\n",
        "        # Normalize by exponent relative to target: r in [0,1]\n",
        "        t = max(2, 2048)  # safety; avoid log2(1)\n",
        "        # Guard: if v is not a power of two or is <1, handle gracefully\n",
        "        try:\n",
        "            r = max(0.0, min(1.0, math.log2(v) / math.log2(t)))\n",
        "        except ValueError:\n",
        "            r = 0.0\n",
        "        idx = int(round(r * (len(GRAD) - 1)))\n",
        "        return f\"\\x1b[38;5;{GRAD[idx]}m\"\n",
        "\n",
        "    def fmt(v: int) -> str:\n",
        "        s = \".\" if (v == 0 and dot_for_zero) else str(v)\n",
        "        s = s.rjust(cell_w)\n",
        "        return color_code(v) + s + (RESET if colors else \"\")\n",
        "\n",
        "    def hline(left: str, mid: str, right: str) -> str:\n",
        "        return left + mid.join(\"─\" * cell_w for _ in range(size)) + right\n",
        "\n",
        "    rows = []\n",
        "    if border:\n",
        "        rows.append(hline(\"┌\", \"┬\", \"┐\"))\n",
        "    for r in range(size):\n",
        "        content = \"│\".join(fmt(v) for v in b[r])\n",
        "        rows.append((\"│\" + content + \"│\") if border else content)\n",
        "        if border:\n",
        "            rows.append(hline(\"└\" if r == size - 1 else \"├\",\n",
        "                            \"┴\" if r == size - 1 else \"┼\",\n",
        "                            \"┘\" if r == size - 1 else \"┤\"))\n",
        "    return \"\\n\".join(rows)"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(render_board(current_state))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DAJE2LUo9oRR",
        "outputId": "5b9fabf9-53eb-401b-d332-380c060dbca0"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can see the `legal_actions` ie what you can take as `[0, 1, 2, 3]` Let's try doing the action `0`."
      ],
      "metadata": {
        "id": "0AhUa4hW-Dji"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b-gSgthFI_wq",
        "outputId": "592c4e13-b213-4d14-f572-ae753c024792"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ],
      "source": [
        "action = OpenSpielAction(action_id = 0, game_name = \"2048\")\n",
        "result = openenv_process.step(action)\n",
        "current_state = result.observation\n",
        "print(render_board(current_state))"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "So it looks like `0` is a move up action! Let's try `1`."
      ],
      "metadata": {
        "id": "lPSNb8-A-iPn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "action = OpenSpielAction(action_id = 1, game_name = \"2048\")\n",
        "result = openenv_process.step(action)\n",
        "current_state = result.observation\n",
        "print(render_board(current_state))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IUel11Tc-oLB",
        "outputId": "3948616f-8c5b-47b1-b8a9-cf359b268de9"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`1` is a move right action. And `2`:"
      ],
      "metadata": {
        "id": "nUlOshVe-qNL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "action = OpenSpielAction(action_id = 2, game_name = \"2048\")\n",
        "result = openenv_process.step(action)\n",
        "current_state = result.observation\n",
        "print(render_board(current_state))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XU09-KA3-sqs",
        "outputId": "558634b5-eb91-45ab-c7a5-813c87857c1d"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "`2` is a move down. And I guess `3` is just move left!"
      ],
      "metadata": {
        "id": "X2r7Zqw9-u-d"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "action = OpenSpielAction(action_id = 3, game_name = \"2048\")\n",
        "result = openenv_process.step(action)\n",
        "current_state = result.observation\n",
        "print(render_board(current_state))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pFgspqn6-zd2",
        "outputId": "395de09a-ef04-4a3d-95cb-d6d33621fb1e"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;51m  4\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can also print the game status which indicates if no more moves are possible, and also the possible actions you can take!"
      ],
      "metadata": {
        "id": "RJP4TsDq-2ft"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MEa2ngmrvfNm",
        "outputId": "5d6c4134-1a89-427f-eef7-8cbe8802bdc0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "False\n",
            "[0, 1, 2]\n"
          ]
        }
      ],
      "source": [
        "print(current_state.done)\n",
        "print(current_state.legal_actions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VR6czU96cpxf"
      },
      "source": [
        "# RL Environment Setup\n",
        "\n",
        "We'll set up a function to accept some strategy that'll emit an action within `0123` and check the game state.\n",
        "\n",
        "We'll also add a timer to only execute the stratgegy for 2 seconds maximum, otherwise it might never terminate!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "tdgjnf-8z_kr"
      },
      "outputs": [],
      "source": [
        "from typing import Callable\n",
        "from unsloth import execute_with_time_limit\n",
        "import itertools\n",
        "\n",
        "def _execute_strategy(strategy, current_state : OpenSpielObservation):\n",
        "    assert callable(strategy)\n",
        "\n",
        "    steps = 0\n",
        "    total_reward = 0\n",
        "    while not current_state.done:\n",
        "        board, size = convert_to_board(current_state)\n",
        "        action = strategy(board)\n",
        "        try:\n",
        "            action = int(action)\n",
        "        except:\n",
        "            return steps, False\n",
        "        steps += 1\n",
        "        if type(action) is not int or action not in current_state.legal_actions:\n",
        "            return steps, max(itertools.chain.from_iterable(board)) == 2048\n",
        "\n",
        "        global port, openenv_process\n",
        "        port, openenv_process = launch_openenv(port, openenv_process)\n",
        "        action = OpenSpielAction(action_id = action, game_name = \"2048\")\n",
        "        result = openenv_process.step(action)\n",
        "        current_state = result.observation\n",
        "        if result.reward is not None:\n",
        "            total_reward += result.reward\n",
        "    return steps, max(itertools.chain.from_iterable(board)) == 2048\n",
        "\n",
        "@execute_with_time_limit(2)\n",
        "def execute_strategy(strategy : Callable, current_state : OpenSpielObservation):\n",
        "    return _execute_strategy(strategy, current_state)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ywh0HizI9ayE"
      },
      "source": [
        "Let's make a generic strategy to just hit `3`. We should expect this generic strategy to fail:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5bkhqoZc0IO8",
        "outputId": "af6db06f-e92e-40d6-cf1a-9bd46233d400"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(11, False)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ],
      "source": [
        "def always_move_left(board):\n",
        "    return 3\n",
        "\n",
        "# Reset OpenEnv to an initial state!\n",
        "port, openenv_process = launch_openenv(port, openenv_process)\n",
        "result = openenv_process.reset()\n",
        "current_state = result.observation\n",
        "try:\n",
        "    steps, if_done = execute_strategy(always_move_left, current_state)\n",
        "except TimeoutError as e:\n",
        "    print(f\"Timed out with error = {str(e)}\")\n",
        "\n",
        "steps, if_done"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dkuHVdB09sgf"
      },
      "source": [
        "To allow longer strategies for GPT-OSS Reinforcement Learning, we shall allow a 5 second timer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "SK-LfzsA9wbW"
      },
      "outputs": [],
      "source": [
        "@execute_with_time_limit(5)\n",
        "def execute_strategy(strategy : Callable, current_state : OpenSpielObservation):\n",
        "    return _execute_strategy(strategy, current_state)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tRhLV_bZMYxy"
      },
      "source": [
        "# Code Execution\n",
        "\n",
        "To execute and create a new Python function, we first have to check if the function does not call other global variables or cheat. This is called `countering reward hacking` since we don't want the function to cheat.\n",
        "\n",
        "For example the below piece of code is fine, since it only imports Python level functions. We use `check_python_modules`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zz80kvg6M4BG",
        "outputId": "2192393f-acbe-4daa-e418-51996cbcdaf5"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Only Python imports? True\n",
            "{'stdlib': ['math', 'typing'], 'non_stdlib': [], 'relative_imports': 0}\n"
          ]
        }
      ],
      "source": [
        "from unsloth import check_python_modules\n",
        "\n",
        "sample = \"\"\"\n",
        "def strategy(board):\n",
        "    import math\n",
        "    from typing import Callable\n",
        "    return \"0\"\n",
        "\"\"\"\n",
        "ok, info = check_python_modules(sample)\n",
        "print(\"Only Python imports?\", ok)\n",
        "print(info)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bZzVWgKQ-VIg"
      },
      "source": [
        "For the below piece of code, since we import `numpy`, we should not allow the execution:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z89Jw1KB-Ux7",
        "outputId": "7a710771-0e68-4b98-da1d-2b8daccd12fd"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Only Python imports? False\n",
            "{'stdlib': [], 'non_stdlib': ['numpy'], 'relative_imports': 0}\n"
          ]
        }
      ],
      "source": [
        "sample = \"\"\"\n",
        "def strategy(board):\n",
        "    from numpy import matmul\n",
        "    return \"0\"\n",
        "\"\"\"\n",
        "ok, info = check_python_modules(sample)\n",
        "print(\"Only Python imports?\", ok)\n",
        "print(info)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SDSrjOTLVyQm"
      },
      "source": [
        "We also disallow global variable access. We'll use Unsloth's `create_locked_down_function` function\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GcmYAmohVqw2",
        "outputId": "4bd24efe-fcd5-469a-8979-c2edf8b25899"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "name 'np' is not defined\n"
          ]
        }
      ],
      "source": [
        "from unsloth import create_locked_down_function\n",
        "function = \"\"\"\n",
        "def import_numpy():\n",
        "    np.matmul\n",
        "    print(\"Success\")\n",
        "\"\"\"\n",
        "f = create_locked_down_function(function)\n",
        "try:\n",
        "    f()\n",
        "except Exception as e:\n",
        "    print(str(e))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5tJKwLUgZsRq",
        "outputId": "39b4e558-fa71-4c0b-aedf-8c347bcae7a0"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "60\n"
          ]
        }
      ],
      "source": [
        "from unsloth import create_locked_down_function\n",
        "function = \"\"\"\n",
        "def add(a, b):\n",
        "    def adder(a):\n",
        "        return a + b\n",
        "    return adder(b) + b\n",
        "\"\"\"\n",
        "f = create_locked_down_function(function)\n",
        "try:\n",
        "    print(f(10, 20))\n",
        "except Exception as e:\n",
        "    print(str(e))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8CzwCyXIPK04"
      },
      "source": [
        "# Data & RL task setup\n",
        "\n",
        "We now have to create a prompt to tell the model to create a strategy for the 2048 game. You can customize this to some other task for another RL task."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "B-2RRE4HMrQO",
        "outputId": "4fd89ae9-605e-42d8-eb94-961df8d7962b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Create a new short 2048 strategy using only native Python code.\n",
            "You are given a list of list of numbers for the current board state.\n",
            "Output one action for \"0\", \"1\", \"2\", \"3\" on what is the optimal next step.\n",
            "Output your new short function in backticks using the format below:\n",
            "```python\n",
            "def strategy(board):\n",
            "    return \"0\" # Example\n",
            "```\n",
            "All helper functions should be inside def strategy. Only output the short function `strategy`.\n"
          ]
        }
      ],
      "source": [
        "prompt = \"\"\"\n",
        "Create a new short 2048 strategy using only native Python code.\n",
        "You are given a list of list of numbers for the current board state.\n",
        "Output one action for \"0\", \"1\", \"2\", \"3\" on what is the optimal next step.\n",
        "Output your new short function in backticks using the format below:\n",
        "```python\n",
        "def strategy(board):\n",
        "    return \"0\" # Example\n",
        "```\n",
        "All helper functions should be inside def strategy. Only output the short function `strategy`.\n",
        "\"\"\".strip()\n",
        "print(prompt)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MIdudFUodN4i"
      },
      "source": [
        "First, let's prompt GPT-OSS without RL and see how it goes:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9HJxrS76h3Ds",
        "outputId": "78a99ef6-d0cb-4787-a9ae-08d98465b508"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<|start|>system<|message|>You are ChatGPT, a large language model trained by OpenAI.\n",
            "Knowledge cutoff: 2024-06\n",
            "Current date: 2025-10-27\n",
            "\n",
            "Reasoning: low\n",
            "\n",
            "# Valid channels: analysis, commentary, final. Channel must be included for every message.\n",
            "Calls to these tools must go to the commentary channel: 'functions'.<|end|><|start|>user<|message|>Create a new short 2048 strategy using only native Python code.\n",
            "You are given a list of list of numbers for the current board state.\n",
            "Output one action for \"0\", \"1\", \"2\", \"3\" on what is the optimal next step.\n",
            "Output your new short function in backticks using the format below:\n",
            "```python\n",
            "def strategy(board):\n",
            "    return \"0\" # Example\n",
            "```\n",
            "All helper functions should be inside def strategy. Only output the short function `strategy`.<|end|><|start|>assistant<|channel|>analysis<|message|>Need to implement simple strategy: maybe random move, or simple heuristic: choose move that results in highest tile. Simplify: evaluate immediate score - sum of resulting board after merge if legal. Choose move with highest reward. Just naive heuristic. Provide code.<|end|><|start|>assistant<|channel|>final<|message|>```python\n",
            "def strategy(board):\n",
            "    import random\n",
            "\n",
            "    def can_move(b, dir):\n",
            "        n = len(b)\n",
            "        if dir == 0:  # up\n",
            "            for j in range(n):\n",
            "                for i in range(1, n):\n",
            "                    if b[i][j] != 0:\n",
            "                        break\n",
            "                else:\n",
            "                    return True\n",
            "        if dir == 1:  # right\n",
            "            for i in range(n):\n",
            "                for j in range(n-2, -1, -1):\n",
            "                    if b[i][j] != 0:\n",
            "                        break\n",
            "                else:\n",
            "                    return True\n",
            "        if dir == 2:  # down\n",
            "            for j in range(n):\n",
            "                for i in range(n-2, -1, -1):\n",
            "                    if b[i][j] != 0:\n",
            "                        break\n",
            "                else:\n",
            "                    return True\n",
            "        if dir == 3:  # left\n",
            "            for i in range(n):\n",
            "                for j in range(1, n):\n",
            "                    if b[i][j] != 0:\n",
            "                        break\n",
            "                else:\n",
            "                    return True\n",
            "        return False\n",
            "\n",
            "    def simulate(b, dir):\n",
            "        b = [row[:] for row in b]\n",
            "        n = len(b)\n",
            "        if dir == 0:  # up\n",
            "            for j in range(n):\n",
            "                pos = 0\n",
            "                prev = None\n",
            "                for i in range(n):\n",
            "                    if b[i][j] == 0:\n",
            "                        continue\n",
            "                    if prev is None:\n",
            "                        prev = b[i][j]\n",
            "                    else:\n",
            "                        if prev == b[i][j]:\n",
            "                            b[pos][j] = prev*2\n",
            "                            prev = None\n",
            "                        else:\n",
            "                            b[pos][j] = prev\n",
            "                            pos += 1\n",
            "                            prev = b[i][j]\n",
            "                if prev is not None:\n",
            "                    b[pos][j] = prev\n",
            "        elif dir == 1:  # right\n",
            "            for i in range(n):\n",
            "                pos = n-1\n",
            "                prev = None\n",
            "                for j in range(n-1, -1, -1):\n",
            "                    if b[i][j] == 0:\n",
            "                        continue\n",
            "\n"
          ]
        }
      ],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 512,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iknaWZNudTNq"
      },
      "source": [
        "# Reward functions\n",
        "\n",
        "We now design a `extract_function` function which simply extracts the function wrapped in 3 back ticks.\n",
        "\n",
        "And 3 reward functions:\n",
        "\n",
        "1. `function_works` which rewards the model if the strategy is a valid Python function.\n",
        "2. `no_cheating` which checks if the function imported other modules, and if it did, we penalize it.\n",
        "3. `strategy_succeeds` which checks if the game strategy actually succeeds in attaining 2048 after running the auto-generated strategy."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8JJGXKdJ-Zl_",
        "outputId": "5fb04cb7-451e-46b2-c964-80a9b4f9bdbe"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "def strategy(board):\n",
            "    return \"0\" # Example\n"
          ]
        }
      ],
      "source": [
        "def extract_function(text):\n",
        "    if text.count(\"```\") >= 2:\n",
        "        first = text.find(\"```\") + 3\n",
        "        second = text.find(\"```\", first)\n",
        "        fx = text[first : second].strip()\n",
        "        fx = fx.removeprefix(\"python\\n\")\n",
        "        fx = fx[fx.find(\"def\"):]\n",
        "        if fx.startswith(\"def strategy(board):\"): return fx\n",
        "    return None\n",
        "print(extract_function(prompt))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KLXEcf_HSJlI"
      },
      "source": [
        "Below is our `function_works` reward function which uses Python's `exec` but guarded by not allowing leakage of local and global variables. We can also use `check_python_modules` first to check if there are errors before even executing the function:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h3-B0IIsS56S",
        "outputId": "73b7e134-3d10-456a-d9d2-c29b5c5d5714"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(False,\n",
              " {'error': \"SyntaxError: expected '(' (<unknown>, line 1)\",\n",
              "  'stdlib': [],\n",
              "  'non_stdlib': [],\n",
              "  'relative_imports': 0})"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "ok, info = check_python_modules(\"def a\")\n",
        "ok, info"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "qgFNXORy-lpO"
      },
      "outputs": [],
      "source": [
        "def function_works(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_python_modules(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            score = -2.0\n",
        "        else:\n",
        "            try:\n",
        "                new_strategy = create_locked_down_function(function)\n",
        "                score = 1.0\n",
        "            except:\n",
        "                score = -0.5\n",
        "        scores.append(score)\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gf69i2WT-m4K"
      },
      "source": [
        "`no_cheating` checks if the function cheated since it might have imported Numpy or other functions:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "cUfHzCVx-nGK"
      },
      "outputs": [],
      "source": [
        "def no_cheating(completions, **kwargs):\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if function is not None:\n",
        "            ok, info = check_python_modules(function)\n",
        "            scores.append(1.0 if ok else -20.0) # Penalize heavily!\n",
        "        else:\n",
        "            scores.append(-1.0) # Failed creating function\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "slnqWG3FTror"
      },
      "source": [
        "Next `strategy_succeeds` checks if the strategy actually allows the game to terminate. Imagine if the strategy simply returned \"0\" which would fail after a time limit of 10 seconds.\n",
        "\n",
        "We also add a global `PRINTER` to print out the strategy and board state."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "sNi129lYTpZ2"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "global PRINTER\n",
        "PRINTER = 0\n",
        "def strategy_succeeds(completions, **kwargs):\n",
        "    global PRINTER\n",
        "    scores = []\n",
        "    for completion in completions:\n",
        "        printed = False\n",
        "        score = 0\n",
        "        response = completion[0][\"content\"]\n",
        "        function = extract_function(response)\n",
        "        if PRINTER % 5 == 0:\n",
        "            printed = True\n",
        "            print(function)\n",
        "        PRINTER += 1\n",
        "        if function is not None:\n",
        "            ok, info = check_python_modules(function)\n",
        "        if function is None or \"error\" in info:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            new_strategy = create_locked_down_function(function)\n",
        "        except:\n",
        "            scores.append(0)\n",
        "            continue\n",
        "        try:\n",
        "            # Reset OpenEnv to an initial state!\n",
        "            global port, openenv_process\n",
        "            port, openenv_process = launch_openenv(port, openenv_process)\n",
        "            result = openenv_process.reset()\n",
        "            current_state = result.observation\n",
        "            steps, if_done = execute_strategy(new_strategy, current_state)\n",
        "            print(f\"Steps = {steps} If Done = {if_done}\")\n",
        "            if printed is False:\n",
        "                print(function)\n",
        "            print(render_board(current_state))\n",
        "            if if_done:\n",
        "                scores.append(20.0) # Success - massively reward!\n",
        "            else:\n",
        "                scores.append(2.0) # Failed but function works!\n",
        "        except TimeoutError as e:\n",
        "            print(\"Timeout\")\n",
        "            scores.append(-1.0) # Failed with timeout\n",
        "        except Exception as e:\n",
        "            print(f\"Exception = {str(e)}\")\n",
        "            scores.append(-3.0) # Failed\n",
        "    return scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TCpSxtvSeAG_"
      },
      "source": [
        "We'll now create the dataset which includes a replica of our prompt. Remember to add a reasoning effort of low! You can choose high reasoning mode, but this'll only work on more memory GPUs like MI300s."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ldf6SjLHVPRv",
        "outputId": "d54b1588-abca-4215-e2fa-6bde90a0a150"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "181\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'prompt': [{'content': 'Create a new short 2048 strategy using only native Python code.\\nYou are given a list of list of numbers for the current board state.\\nOutput one action for \"0\", \"1\", \"2\", \"3\" on what is the optimal next step.\\nOutput your new short function in backticks using the format below:\\n```python\\ndef strategy(board):\\n    return \"0\" # Example\\n```\\nAll helper functions should be inside def strategy. Only output the short function `strategy`.',\n",
              "   'role': 'user'}],\n",
              " 'answer': 0,\n",
              " 'reasoning_effort': 'low'}"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ],
      "source": [
        "from datasets import Dataset\n",
        "dataset = Dataset.from_list([{\"prompt\" : [{\"role\": \"user\", \"content\": prompt.strip()}], \"answer\" : 0, \"reasoning_effort\": \"low\"}]*1000)\n",
        "maximum_length = len(tokenizer.apply_chat_template([{\"role\": \"user\", \"content\": prompt.strip()}], add_generation_prompt = True))\n",
        "print(maximum_length)\n",
        "dataset[0]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9-IOMhVg-2AM"
      },
      "source": [
        "<a name=\"Train\"></a>\n",
        "### Train the model\n",
        "\n",
        "Now set up GRPO Trainer and all configurations! We also support GSPO, GAPO, Dr GRPO and more! Go the Unsloth [Reinforcement Learning Docs](https://docs.unsloth.ai/get-started/reinforcement-learning-rl-guide) for more options.\n",
        "\n",
        "We're also using [TrackIO](https://github.com/gradio-app/trackio) which allows you to visualize all training metrics straight inside the notebook fully locally!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptqkXK2D4d6p",
        "outputId": "b0a81724-588d-4755-9ca4-90addf328e86"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Unsloth: We now expect `per_device_train_batch_size` to be a multiple of `num_generations`.\n",
            "We will change the batch size of 1 to the `num_generations` of 2\n"
          ]
        }
      ],
      "source": [
        "max_prompt_length = maximum_length + 1 # + 1 just in case!\n",
        "max_completion_length = max_seq_length - max_prompt_length\n",
        "\n",
        "from trl import GRPOConfig, GRPOTrainer\n",
        "training_args = GRPOConfig(\n",
        "    temperature = 1.0,\n",
        "    learning_rate = 5e-5,\n",
        "    weight_decay = 0.001,\n",
        "    warmup_ratio = 0.1,\n",
        "    lr_scheduler_type = \"linear\",\n",
        "    optim = \"adamw_8bit\",\n",
        "    logging_steps = 1,\n",
        "    per_device_train_batch_size = 1,\n",
        "    gradient_accumulation_steps = 1, # Increase to 4 for smoother training\n",
        "    num_generations = 2, # Decrease if out of memory\n",
        "    max_prompt_length = max_prompt_length,\n",
        "    max_completion_length = max_completion_length,\n",
        "    # num_train_epochs = 1, # Set to 1 for a full training run\n",
        "    max_steps = 600,\n",
        "    save_steps = 100,\n",
        "    report_to = \"trackio\", # Can use Weights & Biases, TrackIO\n",
        "    output_dir = \"outputs\",\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # fp16_full_eval = True,\n",
        "    # per_device_eval_batch_size = 4,\n",
        "    # eval_accumulation_steps = 1,\n",
        "    # eval_strategy = \"steps\",\n",
        "    # eval_steps = 1,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r9Mv8UZO5hz-"
      },
      "source": [
        "And let's run the trainer! If you scroll up, you'll see a table of rewards. The goal is to see the `reward` column increase!\n",
        "\n",
        "You might have to wait 150 to 200 steps for any action. You'll probably get 0 reward for the first 100 steps. Please be patient!\n",
        "\n",
        "| Step | Training Loss | reward    | reward_std | completion_length | kl       |\n",
        "|------|---------------|-----------|------------|-------------------|----------|\n",
        "| 1    | 0.000000      | 0.125000  | 0.000000   | 200.000000        | 0.000000 |\n",
        "| 2    | 0.000000      | 0.072375  | 0.248112   | 200.000000        | 0.000000 |\n",
        "| 3    | 0.000000      | -0.079000 | 0.163776   | 182.500000        | 0.000005 |\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "id": "vzOuSVCL_GA9"
      },
      "outputs": [],
      "source": [
        "# For optional training + evaluation\n",
        "# new_dataset = dataset.train_test_split(test_size = 0.01)\n",
        "\n",
        "trainer = GRPOTrainer(\n",
        "    model = model,\n",
        "    processing_class = tokenizer,\n",
        "    reward_funcs = [\n",
        "        function_works,\n",
        "        no_cheating,\n",
        "        strategy_succeeds,\n",
        "    ],\n",
        "    args = training_args,\n",
        "    train_dataset = dataset,\n",
        "\n",
        "    # For optional training + evaluation\n",
        "    # train_dataset = new_dataset[\"train\"],\n",
        "    # eval_dataset = new_dataset[\"test\"],\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fQhtuwP4cf34"
      },
      "source": [
        "And let's train the model! **NOTE** This might be quite slow! 600 steps takes ~5 hours or longer.\n",
        "\n",
        "[TrackIO](https://github.com/gradio-app/trackio) might be a bit slow to load - wait 2 minutes until the graphs pop up!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "VGRxPdSCcfC3",
        "outputId": "c51867ef-1c8a-4182-b3ed-da884938d4fb"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The tokenizer has new PAD/BOS/EOS tokens that differ from the model config and generation config. The model config and generation config were aligned accordingly, being updated with the tokenizer's values. Updated tokens: {'bos_token_id': 199998, 'pad_token_id': 200017}.\n",
            "==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1\n",
            "   \\\\   /|    Num examples = 1,000 | Num Epochs = 1 | Total steps = 600\n",
            "O^O/ \\_/ \\    Batch size per device = 2 | Gradient accumulation steps = 1\n",
            "\\        /    Data Parallel GPUs = 1 | Total batch size (2 x 1 x 1) = 2\n",
            " \"-____-\"     Trainable parameters = 1,990,656 of 20,916,747,840 (0.01% trained)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "* Running on public URL: https://81489b1a76036f6928.gradio.live\n",
            "* Trackio project initialized: huggingface\n",
            "* Trackio metrics logged to: /root/.cache/huggingface/trackio\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "<div><iframe src=\"https://81489b1a76036f6928.gradio.live/?project=huggingface&write_token=D2RWzlLArcXWmoPW0DFimbZWy1kNME70fEd-rgUBOgY\" width=\"100%\" height=\"1000px\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "* Created new run: dainty-sunset-0\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "`generation_config` default values have been modified to match model-specific defaults: {'max_length': 131072}. If this is not desired, please set these values explicitly.\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "None\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='14' max='600' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [ 14/600 15:49 < 12:52:29, 0.01 it/s, Epoch 0.01/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>reward</th>\n",
              "      <th>reward_std</th>\n",
              "      <th>completions / mean_length</th>\n",
              "      <th>completions / min_length</th>\n",
              "      <th>completions / max_length</th>\n",
              "      <th>completions / clipped_ratio</th>\n",
              "      <th>completions / mean_terminated_length</th>\n",
              "      <th>completions / min_terminated_length</th>\n",
              "      <th>completions / max_terminated_length</th>\n",
              "      <th>kl</th>\n",
              "      <th>rewards / function_works / mean</th>\n",
              "      <th>rewards / function_works / std</th>\n",
              "      <th>rewards / no_cheating / mean</th>\n",
              "      <th>rewards / no_cheating / std</th>\n",
              "      <th>rewards / strategy_succeeds / mean</th>\n",
              "      <th>rewards / strategy_succeeds / std</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>1</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-12.500000</td>\n",
              "      <td>13.435029</td>\n",
              "      <td>469.000000</td>\n",
              "      <td>352.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>352.000000</td>\n",
              "      <td>352.000000</td>\n",
              "      <td>352.000000</td>\n",
              "      <td>0.003053</td>\n",
              "      <td>-2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-10.500000</td>\n",
              "      <td>13.435029</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.500000</td>\n",
              "      <td>3.535534</td>\n",
              "      <td>438.000000</td>\n",
              "      <td>421.000000</td>\n",
              "      <td>455.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>438.000000</td>\n",
              "      <td>421.000000</td>\n",
              "      <td>455.000000</td>\n",
              "      <td>0.001082</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>3.535534</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>3</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>417.000000</td>\n",
              "      <td>280.000000</td>\n",
              "      <td>554.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>417.000000</td>\n",
              "      <td>280.000000</td>\n",
              "      <td>554.000000</td>\n",
              "      <td>0.006923</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>4</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>4.949748</td>\n",
              "      <td>364.000000</td>\n",
              "      <td>142.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>142.000000</td>\n",
              "      <td>142.000000</td>\n",
              "      <td>142.000000</td>\n",
              "      <td>0.009904</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>5</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-3.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.001758</td>\n",
              "      <td>-2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>6</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>437.500000</td>\n",
              "      <td>350.000000</td>\n",
              "      <td>525.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>437.500000</td>\n",
              "      <td>350.000000</td>\n",
              "      <td>525.000000</td>\n",
              "      <td>0.002351</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>7</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>333.000000</td>\n",
              "      <td>99.000000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>333.000000</td>\n",
              "      <td>99.000000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>0.006251</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>8</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>4.949748</td>\n",
              "      <td>576.500000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>567.000000</td>\n",
              "      <td>0.001354</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>9</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>4.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>519.500000</td>\n",
              "      <td>453.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>453.000000</td>\n",
              "      <td>453.000000</td>\n",
              "      <td>453.000000</td>\n",
              "      <td>0.003413</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>2.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>10</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>4.949748</td>\n",
              "      <td>464.000000</td>\n",
              "      <td>342.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>342.000000</td>\n",
              "      <td>342.000000</td>\n",
              "      <td>342.000000</td>\n",
              "      <td>0.008979</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>11</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>4.949748</td>\n",
              "      <td>576.000000</td>\n",
              "      <td>566.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>566.000000</td>\n",
              "      <td>566.000000</td>\n",
              "      <td>566.000000</td>\n",
              "      <td>0.001162</td>\n",
              "      <td>-0.500000</td>\n",
              "      <td>2.121320</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>1.414214</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>12</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>-1.250000</td>\n",
              "      <td>2.474874</td>\n",
              "      <td>548.000000</td>\n",
              "      <td>510.000000</td>\n",
              "      <td>586.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>510.000000</td>\n",
              "      <td>510.000000</td>\n",
              "      <td>510.000000</td>\n",
              "      <td>0.001801</td>\n",
              "      <td>-1.250000</td>\n",
              "      <td>1.060660</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>1.414214</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Unsloth: Will smartly offload gradients to save VRAM!\n",
            "Steps = 25 If Done = False\n",
            "def strategy(board):\n",
            "    # simple heuristic: pick move that keeps the board most unchanged\n",
            "    moves = {'0':0,'1':0,'2':0,'3':0}\n",
            "    # try each move, compute resulting score (max tile + sum)\n",
            "    def move(board, direction):\n",
            "        import copy\n",
            "        size = len(board)\n",
            "        new = [[0]*size for _ in range(size)]\n",
            "        for i in range(size):\n",
            "            line = []\n",
            "            if direction in (1,3):\n",
            "                line = board[i]\n",
            "            else:\n",
            "                line = [board[j][i] for j in range(size)]\n",
            "            if direction in (2,3):\n",
            "                line = line[::-1]\n",
            "            merged = []\n",
            "            skip = False\n",
            "            for j, val in enumerate(line):\n",
            "                if skip:\n",
            "                    skip = False\n",
            "                    continue\n",
            "                if val == 0:\n",
            "                    continue\n",
            "                if j+1 < len(line) and line[j+1] == val:\n",
            "                    merged.append(val*2)\n",
            "                    skip = True\n",
            "                else:\n",
            "                    merged.append(val)\n",
            "            while len(merged) < size:\n",
            "                merged.append(0)\n",
            "            if direction in (2,3):\n",
            "                merged = merged[::-1]\n",
            "            if direction in (1,3):\n",
            "                for k in range(size):\n",
            "                    new[i][k] = merged[k]\n",
            "            else:\n",
            "                for k in range(size):\n",
            "                    new[k][i] = merged[k]\n",
            "        return new\n",
            "    best = None\n",
            "    best_val = -1\n",
            "    for d in ('0','1','2','3'):\n",
            "        nb = move(board, int(d))\n",
            "        val = max(max(row) for row in nb)\n",
            "        if val > best_val:\n",
            "            best_val = val\n",
            "            best = d\n",
            "    return best\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Exception = list index out of range\n",
            "Steps = 1 If Done = False\n",
            "def strategy(board):\n",
            "    import copy\n",
            "    # helper to move a row left\n",
            "    def compress(row):\n",
            "        new = [x for x in row if x != 0]\n",
            "        for i in range(len(new)-1):\n",
            "            if new[i]==new[i+1]:\n",
            "                new[i]*=2\n",
            "                new[i+1]=0\n",
            "        new = [x for x in new if x!=0]\n",
            "        return new + [0]*(len(row)-len(new))\n",
            "    # perform move with given direction\n",
            "    def move(board, dir):\n",
            "        new_board=copy.deepcopy(board)\n",
            "        if dir==0:          # left\n",
            "            for i in range(4):\n",
            "                new_board[i]=compress(new_board[i])\n",
            "        elif dir==1:        # right\n",
            "            for i in range(4):\n",
            "                new_board[i]=list(reversed(compress(list(reversed(new_board[i])))))\n",
            "        elif dir==2:        # up\n",
            "            for j in range(4):\n",
            "                col=[new_board[i][j] for i in range(4)]\n",
            "                new_col=compress(col)\n",
            "                for i in range(4):\n",
            "                    new_board[i][j]=new_col[i]\n",
            "        elif dir==3:        # down\n",
            "            for j in range(4):\n",
            "                col=list(reversed([new_board[i][j] for i in range(4)]))\n",
            "                new_col=list(reversed(compress(col)))\n",
            "                for i in range(4):\n",
            "                    new_board[i][j]=new_col[i]\n",
            "        return new_board\n",
            "    # evaluate board by sum of squares (higher is better)\n",
            "    def score(b):\n",
            "        return sum(x*x for row in b for x in row)\n",
            "    best_dir=None; best_score=-1\n",
            "    for d in range(4):\n",
            "        nb=move(board,d)\n",
            "        if any(row.count(0)==0 for row in nb): pass  # ignore full board\n",
            "        sc=score(nb)\n",
            "        if sc>best_score:\n",
            "            best_score=sc; best_dir=d\n",
            "    return str(best_dir)\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "def strategy(board):\n",
            "    # Simple heuristic: move the largest tile towards a corner\n",
            "    size = len(board)\n",
            "    moves = []\n",
            "    # compute sums of rows and columns\n",
            "    row_sums = [sum(r) for r in board]\n",
            "    col_sums = [sum(board[i][j] for i in range(size)) for j in range(size)]\n",
            "    # choose direction that keeps highest tile in corner\n",
            "    best = None\n",
            "    best_val = -1\n",
            "    for d in range(4):\n",
            "        if d == 0:  # up\n",
            "            val = max(row_sums[0], row_sums[1], row_sums[2], row_sums[3]) # not accurate, placeholder\n",
            "        if d == 1:  # down\n",
            "            val = sum(row_sums)\n",
            "        if d == 2:  # left\n",
            "            val = sum(col_sums)\n",
            "        if d == 3:  # right\n",
            "            val = sum(col_sums)\n",
            "        # take the first direction as a simple placeholder\n",
            "        return \"0\"\n",
            "Steps = 1 If Done = False\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 4 If Done = False\n",
            "def strategy(board):\n",
            "    # Simple heuristic: move left if possible, otherwise right, then up, then down.\n",
            "    for row in board:\n",
            "        if row[0] == 0:\n",
            "            return \"0\"\n",
            "    for row in board:\n",
            "        if row[3] == 0:\n",
            "            return \"1\"\n",
            "    for row in board:\n",
            "        if row[0] == 0 or row[3] == 0:\n",
            "            return \"2\"\n",
            "    return \"3\"\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "def strategy(board):\n",
            "    # Easy heuristic: try to push merges toward the right\n",
            "    def score_move(dir):\n",
            "        from copy import deepcopy\n",
            "        b = deepcopy(board)\n",
            "        def move_left(b):\n",
            "            new=[]\n",
            "            for row in b:\n",
            "                nonZero=[x for x in row if x]\n",
            "                merged=[]\n",
            "                i=0\n",
            "                while i<len(nonZero):\n",
            "                    if i+1<len(nonZero) and nonZero[i]==nonZero[i+1]:\n",
            "                        merged.append(nonZero[i]*2); i+=2\n",
            "                    else:\n",
            "                        merged.append(nonZero[i]); i+=1\n",
            "                merged+= [0]*(len(row)-len(merged))\n",
            "                new.append(merged)\n",
            "            return new\n",
            "        if dir==\"0\": # left\n",
            "            return move_left(b)\n",
            "        # right\n",
            "        b=[list(reversed(r)) for r in b]\n",
            "        b=move_left(b)\n",
            "        return [list(reversed(r)) for r in b]\n",
            "    best_dir=\"0\"\n",
            "    max_score=-1\n",
            "    for d in (\"0\",\"1\",\"2\",\"3\"):\n",
            "        new=score_move(d)\n",
            "        s=sum(sum(row) for row in new)\n",
            "        if s>max_score:\n",
            "            max_score=s; best_dir=d\n",
            "    return best_dir\n",
            "Steps = 23 If Done = False\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 2 If Done = False\n",
            "def strategy(board):\n",
            "    # Helper: apply a move and return new board\n",
            "    def move(board, dir):\n",
            "        N = len(board)\n",
            "        def compress(row):\n",
            "            new = [i for i in row if i != 0]\n",
            "            res = []\n",
            "            i = 0\n",
            "            while i < len(new):\n",
            "                if i+1 < len(new) and new[i] == new[i+1]:\n",
            "                    res.append(new[i]*2)\n",
            "                    i += 2\n",
            "                else:\n",
            "                    res.append(new[i])\n",
            "                    i += 1\n",
            "            res += [0]*(N-len(res))\n",
            "            return res\n",
            "        if dir==0:  # up\n",
            "            res=[[0]*N for _ in range(N)]\n",
            "            for j in range(N):\n",
            "                col=[board[i][j] for i in range(N)]\n",
            "                newcol=compress(col)\n",
            "                for i in range(N):\n",
            "                    res[i][j]=newcol[i]\n",
            "        elif dir==1:  # down\n",
            "            res=[[0]*N for _ in range(N)]\n",
            "            for j in range(N):\n",
            "                col=[board[i][j] for i in range(N)][::-1]\n",
            "                newcol=compress(col)\n",
            "                newcol=newcol[::-1]\n",
            "                for i in range(N):\n",
            "                    res[i][j]=newcol[i]\n",
            "        elif dir==2:  # left\n",
            "            res=[[0]*N for _ in range(N)]\n",
            "            for i in range(N):\n",
            "                newrow=compress(board[i])\n",
            "                for j in range(N):\n",
            "                    res[i][j]=newrow[j]\n",
            "        else:  # right\n",
            "            res=[[0]*N for _ in range(N)]\n",
            "            for i in range(N):\n",
            "                row=board[i][::-1]\n",
            "                newrow=compress(row)\n",
            "                newrow=newrow[::-1]\n",
            "                for j in range(N):\n",
            "                    res[i][j]=newrow[j]\n",
            "        return res\n",
            "\n",
            "    # Heuristic: choose move with highest sum after move\n",
            "    best_dir=None\n",
            "    best_score=-1\n",
            "    for d in range(4):\n",
            "        new=move(board,d)\n",
            "        score=sum(sum(row) for row in new)\n",
            "        if score>best_score:\n",
            "            best_score=score\n",
            "            best_dir=d\n",
            "    return str(best_dir)\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 8 If Done = False\n",
            "def strategy(board):\n",
            "    # Simple heuristic: always move left if possible, else right\n",
            "    for i, row in enumerate(board):\n",
            "        for j, val in enumerate(row):\n",
            "            if val == 0:  # Found empty cell\n",
            "                return \"0\"\n",
            "    # If board is full, default to right\n",
            "    return \"2\"\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 5 If Done = False\n",
            "def strategy(board):\n",
            "    # Simple heuristic: pick the first available move with the least cumulative difference\n",
            "    moves = [0, 1, 2, 3]\n",
            "    best_move = moves[0]\n",
            "    best_score = float('inf')\n",
            "    for move in moves:\n",
            "        # simulate move by moving tiles in that direction (simplified)\n",
            "        new_board = [row[:] for row in board]\n",
            "        score = 0\n",
            "        if move == 0:  # up\n",
            "            for j in range(len(board[0])):\n",
            "                merged = []\n",
            "                for i in range(len(new_board)):\n",
            "                    if new_board[i][j] != 0:\n",
            "                        merged.append(new_board[i][j])\n",
            "                merged += [0]*(len(board)-len(merged))\n",
            "                for i in range(len(new_board)):\n",
            "                    new_board[i][j] = merged[i]\n",
            "        elif move == 1:  # right\n",
            "            for i in range(len(new_board)):\n",
            "                merged = []\n",
            "                for j in range(len(new_board[0])):\n",
            "                    if new_board[i][j] != 0:\n",
            "                        merged.append(new_board[i][j])\n",
            "                merged += [0]*(len(board)-len(merged))\n",
            "                for j in range(len(new_board[0])):\n",
            "                    new_board[i][j] = merged[j]\n",
            "        elif move == 2:  # down\n",
            "            for j in range(len(new_board[0])):\n",
            "                merged = []\n",
            "                for i in range(len(new_board)-1, -1, -1):\n",
            "                    if new_board[i][j] != 0:\n",
            "                        merged.append(new_board[i][j])\n",
            "                merged += [0]*(len(board)-len(merged))\n",
            "                for i in range(len(new_board)):\n",
            "                    new_board[i][j] = merged[len(new_board)-i-1]\n",
            "        elif move == 3:  # left\n",
            "            for i in range(len(new_board)):\n",
            "                merged = []\n",
            "                for j in range(len(new_board[0])):\n",
            "                    if new_board[i][j] != 0:\n",
            "                        merged.append(new_board[i][j])\n",
            "                merged += [0]*(len(board)-len(merged))\n",
            "                for j in range(len(new_board[0])):\n",
            "                    new_board[i][j] = merged[j]\n",
            "        # evaluate score as sum of differences from max tile\n",
            "        score = sum(sum(row) for row in new_board)\n",
            "        if score < best_score:\n",
            "            best_score = score\n",
            "            best_move = move\n",
            "    return str(best_move)\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "def strategy(board):\n",
            "    # Define move simulation\n",
            "    def move(board, dir):\n",
            "        n = len(board)\n",
            "        moved = [[0]*n for _ in range(n)]\n",
            "        if dir==0:  # up\n",
            "            for j in range(n):\n",
            "                col=[board[i][j] for i in range(n) if board[i][j]]\n",
            "                up=[0]*n; k=0\n",
            "                for v in col:\n",
            "                    if up[k]==v:\n",
            "                        up[k]*=2\n",
            "                    else:\n",
            "                        up[k]=v\n",
            "                        k+=1\n",
            "                for i in range(n): moved[i][j]=up[i]\n",
            "        elif dir==1:  # down\n",
            "            for j in range(n):\n",
            "                col=[board[i][j] for i in range(n-1,-1,-1) if board[i][j]]\n",
            "                down=[0]*n; k=n-1\n",
            "                for v in col:\n",
            "                    if down[k]==v:\n",
            "                        down[k]*=2\n",
            "                    else:\n",
            "                        down[k]=v\n",
            "                        k-=1\n",
            "                for i in range(n): moved[i][j]=down[i]\n",
            "        elif dir==2:  # left\n",
            "            for i in range(n):\n",
            "                row=[board[i][j] for j in range(n) if board[i][j]]\n",
            "                left=[0]*n; k=0\n",
            "                for v in row:\n",
            "                    if left[k]==v:\n",
            "                        left[k]*=2\n",
            "                    else:\n",
            "                        left[k]=v\n",
            "                        k+=1\n",
            "                moved[i]=left\n",
            "        else:  # right\n",
            "            for i in range(n):\n",
            "                row=[board[i][j] for j in range(n-1,-1,-1) if board[i][j]]\n",
            "                right=[0]*n; k=n-1\n",
            "                for v in row:\n",
            "                    if right[k]==v:\n",
            "                        right[k]*=2\n",
            "                    else:\n",
            "                        right[k]=v\n",
            "                        k-=1\n",
            "                moved[i]=right\n",
            "        return moved\n",
            "\n",
            "    # Evaluate a move by depth‑1 simulation (choose highest tile after move)\n",
            "    best_dir=None\n",
            "    best_val=-1\n",
            "    for d in range(4):\n",
            "        new=move(board,d)\n",
            "        best_tile=max(max(row) for row in new)\n",
            "        if best_tile>best_val:\n",
            "            best_val=best_tile\n",
            "            best_dir=str(d)\n",
            "    return best_dir\n",
            "Steps = 4 If Done = False\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 7 If Done = False\n",
            "def strategy(board):\n",
            "    best = None\n",
            "    for move in [\"0\",\"1\",\"2\",\"3\"]:\n",
            "        b = [row[:] for row in board]\n",
            "        n = len(b)\n",
            "        merged = False\n",
            "        if move == \"0\":\n",
            "            for r in range(n):\n",
            "                c = 0\n",
            "                while c < n-1:\n",
            "                    if b[r][c] == b[r][c+1] and b[r][c] != 0:\n",
            "                        b[r][c] *= 2\n",
            "                        b[r][c+1] = 0\n",
            "                        merged = True\n",
            "                    c += 1\n",
            "        elif move == \"1\":\n",
            "            for r in range(n):\n",
            "                c = n-1\n",
            "                while c > 0:\n",
            "                    if b[r][c] == b[r][c-1] and b[r][c] != 0:\n",
            "                        b[r][c] *= 2\n",
            "                        b[r][c-1] = 0\n",
            "                        merged = True\n",
            "                    c -= 1\n",
            "        elif move == \"2\":\n",
            "            for c in range(n):\n",
            "                r = 0\n",
            "                while r < n-1:\n",
            "                    if b[r][c] == b[r+1][c] and b[r][c] != 0:\n",
            "                        b[r][c] *= 2\n",
            "                        b[r+1][c] = 0\n",
            "                        merged = True\n",
            "                    r += 1\n",
            "        else:\n",
            "            for c in range(n):\n",
            "                r = n-1\n",
            "                while r > 0:\n",
            "                    if b[r][c] == b[r-1][c] and b[r][c] != 0:\n",
            "                        b[r][c] *= 2\n",
            "                        b[r-1][c] = 0\n",
            "                        merged = True\n",
            "                    r -= 1\n",
            "        score = sum(sum(row) for row in b)\n",
            "        if best is None or score > best[0]:\n",
            "            best = (score, move)\n",
            "    return best[1]\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 2 If Done = False\n",
            "def strategy(board):\n",
            "    # Define board size and the action mapping\n",
            "    N = 4\n",
            "\n",
            "    # Helper to perform a move; returns a new board\n",
            "    def move(b, dir):\n",
            "        new = [[0]*N for _ in range(N)]\n",
            "        for i in range(N):\n",
            "            # extract and compact the line\n",
            "            line = [b[i][j] for j in range(N) if b[i][j] != 0] if dir==3 else \\\n",
            "                    [b[j][i] for j in range(N) if b[j][i] != 0] if dir==2 else \\\n",
            "                    [b[j][i] for j in range(N) if b[j][i] != 0] if dir==1 else \\\n",
            "                    [b[i][j] for j in range(N) if b[i][j] != 0]\n",
            "            comp = []\n",
            "            idx = 0\n",
            "            while idx < len(line):\n",
            "                if idx+1 < len(line) and line[idx]==line[idx+1]:\n",
            "                    comp.append(line[idx]*2)\n",
            "                    idx += 2\n",
            "                else:\n",
            "                    comp.append(line[idx])\n",
            "                    idx += 1\n",
            "            # fill remaining with zeros\n",
            "            comp += [0]*(N-len(comp))\n",
            "            # write back to new board\n",
            "            for k, val in enumerate(comp):\n",
            "                if dir==0:  # up\n",
            "                    new[k][i] = val\n",
            "                elif dir==1:  # right\n",
            "                    new[i][N-1-k] = val\n",
            "                elif dir==2:  # down\n",
            "                    new[N-1-k][i] = val\n",
            "                else:          # left\n",
            "                    new[i][k] = val\n",
            "        return new\n",
            "\n",
            "    # Evaluate board value: higher is better\n",
            "    def score(b):\n",
            "        s = 0\n",
            "        for row in b:\n",
            "            for v in row:\n",
            "                s += v\n",
            "        return s\n",
            "\n",
            "    best = -1\n",
            "    best_dir = 0\n",
            "    for d in range(4):\n",
            "        new_b = move(board, d)\n",
            "        if new_b == board:\n",
            "            continue  # move had no effect\n",
            "        val = score(new_b)\n",
            "        if val > best:\n",
            "            best = val\n",
            "            best_dir = d\n",
            "    return str(best_dir)\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 4 If Done = False\n",
            "def strategy(board):\n",
            "    # Simple heuristic: prefer moving up (0) if any tile can move up,\n",
            "    # otherwise prefer left (1), then down (2), then right (3).\n",
            "    size = len(board)\n",
            "    def can_move(direction):\n",
            "        for i in range(size):\n",
            "            for j in range(size):\n",
            "                val = board[i][j]\n",
            "                if val == 0:\n",
            "                    continue\n",
            "                if direction == 0:  # up\n",
            "                    if i == 0 or board[i-1][j] != 0:\n",
            "                        continue\n",
            "                    return True\n",
            "                if direction == 1:  # left\n",
            "                    if j == 0 or board[i][j-1] != 0:\n",
            "                        continue\n",
            "                    return True\n",
            "                if direction == 2:  # down\n",
            "                    if i == size-1 or board[i+1][j] != 0:\n",
            "                        continue\n",
            "                    return True\n",
            "                if direction == 3:  # right\n",
            "                    if j == size-1 or board[i][j+1] != 0:\n",
            "                        continue\n",
            "                    return True\n",
            "        return False\n",
            "\n",
            "    for d in range(4):\n",
            "        if can_move(d):\n",
            "            return str(d)\n",
            "    return \"0\"\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "def strategy(board):\n",
            "    # board: list of lists, 4x4 integers (0 for empty)\n",
            "    # Simple greedy heuristic: favor moves that increase the score and keep high tiles in a corner\n",
            "    moves = [\"0\", \"1\", \"2\", \"3\"]  # 0: up, 1: down, 2: left, 3: right\n",
            "\n",
            "    def score(b, move):\n",
            "        # simulate move\n",
            "        def shift(row):\n",
            "            new = [x for x in row if x != 0]\n",
            "            res = []\n",
            "            i = 0\n",
            "            while i < len(new):\n",
            "                if i+1 < len(new) and new[i] == new[i+1]:\n",
            "                    res.append(new[i]*2); i+=2\n",
            "                else:\n",
            "                    res.append(new[i]); i+=1\n",
            "            res += [0]*(4-len(res))\n",
            "            return res\n",
            "        if move == \"0\":  # up\n",
            "            new = [[0]*4 for _ in range(4)]\n",
            "            for c in range(4):\n",
            "                col = [board[r][c] for r in range(4)]\n",
            "                merged = shift(col)\n",
            "                for r in range(4): new[r][c] = merged[r]\n",
            "        elif move == \"1\":  # down\n",
            "            new = [[0]*4 for _ in range(4)]\n",
            "            for c in range(4):\n",
            "                col = [board[r][c] for r in reversed(range(4))]\n",
            "                merged = shift(col)\n",
            "                for i,r in enumerate(reversed(range(4))):\n",
            "                    new[r][c] = merged[i]\n",
            "        elif move == \"2\":  # left\n",
            "            new = [[0]*4 for _ in range(4)]\n",
            "            for r in range(4):\n",
            "                row = board[r]\n",
            "                merged = shift(row)\n",
            "                new[r] = merged\n",
            "        else:  # right\n",
            "            new = [[0]*4 for _ in range(4)]\n",
            "            for r in range(4):\n",
            "                row = list(reversed(board[r]))\n",
            "                merged = shift(row)\n",
            "                for i,c in enumerate(reversed(range(4))):\n",
            "                    new[r][c] = merged[i]\n",
            "        # evaluate result\n",
            "        max_tile = max(max(row) for row in new)\n",
            "        empty = sum(row.count(0) for row in new)\n",
            "        # simple score: higher max tile and more empties\n",
            "        return max_tile*100 + empty\n",
            "    best_move = max(moves, key=lambda m: score(board, m))\n",
            "    return best_move\n",
            "Steps = 11 If Done = False\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "Steps = 4 If Done = False\n",
            "def strategy(board):\n",
            "    # simulate a move and pick the one that keeps most tiles unchanged\n",
            "    moves = {}\n",
            "    for m in \"0123\":\n",
            "        nxt = board[:]\n",
            "        for i in range(4):\n",
            "            line = [x for x in nxt[i] if x]\n",
            "            line += [0]*(4-len(line))\n",
            "            if m==\"0\":\n",
            "                nxt[i] = line\n",
            "            elif m==\"1\":\n",
            "                nxt[i] = line[::-1]\n",
            "            elif m==\"2\":\n",
            "                for j in range(4):\n",
            "                    col = [nxt[k][j] for k in range(4) if nxt[k][j]]\n",
            "                    col += [0]*(4-len(col))\n",
            "                    for k in range(4):\n",
            "                        nxt[k][j] = col[k]\n",
            "            else: # m==\"3\"\n",
            "                for j in range(4):\n",
            "                    col = [nxt[k][j] for k in range(4) if nxt[k][j]]\n",
            "                    col += [0]*(4-len(col))\n",
            "                    col.reverse()\n",
            "                    for k in range(4):\n",
            "                        nxt[k][j] = col[k]\n",
            "        # score = sum of all non-zero tiles\n",
            "        score = sum(x for row in nxt for x in row if x)\n",
            "        moves[m] = score\n",
            "    # choose move that maximizes score\n",
            "    best = max(moves, key=moves.get)\n",
            "    return best\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n",
            "def strategy(board):\n",
            "    # Compute total number of moves available\n",
            "    moves = 0\n",
            "    my_total = 0\n",
            "    for i in range(4):\n",
            "        for j in range(4):\n",
            "            val = board[i][j]\n",
            "            if val == 0:\n",
            "                moves += 1\n",
            "                continue\n",
            "            # Count possible merges with neighbors\n",
            "            for di,dj in ((1,0),(-1,0),(0,1),(0,-1)):\n",
            "                ni, nj = i+di, j+dj\n",
            "                if 0 <= ni < 4 and 0 <= nj < 4:\n",
            "                    if board[ni][nj] == val:\n",
            "                        moves += 1\n",
            "                        break\n",
            "            # Sum of values\n",
            "            my_total += val\n",
            "    # Easy heuristic: if most of the board is empty, push up (0)\n",
            "    if moves > 40:\n",
            "        return \"0\"\n",
            "    # If sum is large, try to push right (1)\n",
            "    if my_total > 2000:\n",
            "        return \"1\"\n",
            "    # If empty cells few, push down (2)\n",
            "    if moves < 20:\n",
            "        return \"2\"\n",
            "    # Default to left (3)\n",
            "    return \"3\"\n",
            "Steps = 4 If Done = False\n",
            "┌───┬───┬───┬───┐\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;51m  4\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "├───┼───┼───┼───┤\n",
            "│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;45m  2\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\u001b[38;5;239m  .\u001b[0m│\n",
            "└───┴───┴───┴───┘\n"
          ]
        }
      ],
      "source": [
        "trainer.train()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tlaUdxC_VHpz"
      },
      "source": [
        "<a name=\"Inference\"></a>\n",
        "# Inference\n",
        "Now let's try the model we just trained!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TwZygRdWf8ab"
      },
      "outputs": [],
      "source": [
        "text = tokenizer.apply_chat_template(\n",
        "    [{\"role\": \"user\", \"content\": prompt}],\n",
        "    tokenize = False,\n",
        "    add_generation_prompt = True,\n",
        "    reasoning_effort = \"low\",\n",
        ")\n",
        "\n",
        "from transformers import TextStreamer\n",
        "_ = model.generate(\n",
        "    **tokenizer(text, return_tensors = \"pt\").to(\"cuda\"),\n",
        "    temperature = 1.0,\n",
        "    max_new_tokens = 1024,\n",
        "    streamer = TextStreamer(tokenizer, skip_prompt = False),\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-NUEmHFSYNTp"
      },
      "source": [
        "<a name=\"Save\"></a>\n",
        "### Saving to float16 or `MXFP4`\n",
        "\n",
        "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `mxfp4` for MXFP4 (OpenAI's GPT-OSS native precision). We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "NjXGTkp7YNtB"
      },
      "outputs": [],
      "source": [
        "# Merge and push to hub in mxfp4 4bit format\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"mxfp4\")\n",
        "if False:\n",
        "    model.push_to_hub_merged(\"repo_id/repo_name\", tokenizer, token = \"hf...\", save_method = \"mxfp4\")\n",
        "\n",
        "# Merge and push to hub in 16bit\n",
        "if False:\n",
        "    model.save_pretrained_merged(\"finetuned_model\", tokenizer, save_method = \"merged_16bit\")\n",
        "if False: # Pushing to HF Hub\n",
        "    model.push_to_hub_merged(\"hf/gpt-oss-finetune\", tokenizer, save_method = \"merged_16bit\", token = \"\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V15Yhj1V9lwG"
      },
      "source": [
        "# And we're done!\n",
        "Congratulations you just learned how to do reinforcement learning with GPT-OSS! There were some advanced topics explained in this notebook - to learn more about GPT-OSS and RL, there are more docs in Unsloth's [Reinforcement Learning Guide with GPT-OSS](https://docs.unsloth.ai/new/gpt-oss-reinforcement-learning)\n",
        "\n",
        "This notebook and all Unsloth notebooks are licensed [LGPL-3.0](https://github.com/unslothai/notebooks?tab=LGPL-3.0-1-ov-file#readme)."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "A100",
      "provenance": [],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "83cdc795bf81400787457cb72ccd265c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bee5c215c01349a893277200730f9a2a",
              "IPY_MODEL_39b720a342264a3ca74ec101010671bb",
              "IPY_MODEL_88a111b1d64d4f2688c65ed28a06c62d"
            ],
            "layout": "IPY_MODEL_c0bf03dbb0bc4e808268fbf8d458fed3"
          }
        },
        "bee5c215c01349a893277200730f9a2a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_90b4a7e0316d4e2fb9e976ec0c61eb77",
            "placeholder": "​",
            "style": "IPY_MODEL_b4248f48607144cd835327991f85d0c5",
            "value": "model.safetensors.index.json: "
          }
        },
        "39b720a342264a3ca74ec101010671bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_337698676cef42fd8b9d356ca9023be4",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a61a310b048e48c4a6434d7e6225f630",
            "value": 1
          }
        },
        "88a111b1d64d4f2688c65ed28a06c62d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_140b6a185bfa433fb48a1c878857b216",
            "placeholder": "​",
            "style": "IPY_MODEL_a654b9b92cd54916bc9ab45fce6fb77a",
            "value": " 1.19M/? [00:00&lt;00:00, 104MB/s]"
          }
        },
        "c0bf03dbb0bc4e808268fbf8d458fed3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "90b4a7e0316d4e2fb9e976ec0c61eb77": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b4248f48607144cd835327991f85d0c5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "337698676cef42fd8b9d356ca9023be4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "a61a310b048e48c4a6434d7e6225f630": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "140b6a185bfa433fb48a1c878857b216": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a654b9b92cd54916bc9ab45fce6fb77a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a16edcdf8b6a4339a5f1c6fe14f35cb1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_88001b08f87a4c5aad2fae41346d93ce",
              "IPY_MODEL_8f5a4260c16441f489c4326ed498bed9",
              "IPY_MODEL_191bfb34de6c4e66b0f4debbd137aa61"
            ],
            "layout": "IPY_MODEL_e7043a72b1be479ea8956228f4a8774c"
          }
        },
        "88001b08f87a4c5aad2fae41346d93ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_082e8f0f8492483a94c8168528950efe",
            "placeholder": "​",
            "style": "IPY_MODEL_b0ae96c9075b441f83eaa019a58c3b30",
            "value": "model-00001-of-00004.safetensors: 100%"
          }
        },
        "8f5a4260c16441f489c4326ed498bed9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_209256542eb6497689ee291cc3180c01",
            "max": 3998751275,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_1b66d8f404854c8da03702634d301a02",
            "value": 3998751275
          }
        },
        "191bfb34de6c4e66b0f4debbd137aa61": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1a5f521f88474cd4b7942aea0f8fa888",
            "placeholder": "​",
            "style": "IPY_MODEL_16c140e98542415fb6b07f8a892185b2",
            "value": " 4.00G/4.00G [00:05&lt;00:00, 601MB/s]"
          }
        },
        "e7043a72b1be479ea8956228f4a8774c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "082e8f0f8492483a94c8168528950efe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b0ae96c9075b441f83eaa019a58c3b30": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "209256542eb6497689ee291cc3180c01": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1b66d8f404854c8da03702634d301a02": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1a5f521f88474cd4b7942aea0f8fa888": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "16c140e98542415fb6b07f8a892185b2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f18bbbacbcd54c9abab52072c3f73c2d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_872cbc648fd1484793b3b763a6a47244",
              "IPY_MODEL_364c26a8fda442d1b8b0a752934963ce",
              "IPY_MODEL_ad78fc1270924bb8a94e02b42cf8ff7c"
            ],
            "layout": "IPY_MODEL_a90ba7ea60434c66b23261547497dce3"
          }
        },
        "872cbc648fd1484793b3b763a6a47244": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_eb9940d6e1d047e49c6f3536c8f3ee30",
            "placeholder": "​",
            "style": "IPY_MODEL_0e77b625c6a2448580275e7009be21c0",
            "value": "model-00002-of-00004.safetensors: 100%"
          }
        },
        "364c26a8fda442d1b8b0a752934963ce": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3b4e60df969940af9baf165996544914",
            "max": 3996690997,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b3ed99825d3043d097caa08756471d7f",
            "value": 3996690997
          }
        },
        "ad78fc1270924bb8a94e02b42cf8ff7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d88119cb71894de6b38cbfd0abd736db",
            "placeholder": "​",
            "style": "IPY_MODEL_32d3ca1e8e26404ea2eff4c6ae746591",
            "value": " 4.00G/4.00G [00:10&lt;00:00, 840MB/s]"
          }
        },
        "a90ba7ea60434c66b23261547497dce3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb9940d6e1d047e49c6f3536c8f3ee30": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0e77b625c6a2448580275e7009be21c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3b4e60df969940af9baf165996544914": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b3ed99825d3043d097caa08756471d7f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d88119cb71894de6b38cbfd0abd736db": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "32d3ca1e8e26404ea2eff4c6ae746591": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "33364a388b964f44bf5503476758ff84": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_bed86bfe5355486d97cb20137de2901e",
              "IPY_MODEL_919f622c4b594e2499103632a8bbe93f",
              "IPY_MODEL_2f9d7a4281124747b4982683cbf5749e"
            ],
            "layout": "IPY_MODEL_dfa095324480404db4b2afa2fa2eadd7"
          }
        },
        "bed86bfe5355486d97cb20137de2901e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1a8ea075a0c7495ca65a6e0c7a04b6d1",
            "placeholder": "​",
            "style": "IPY_MODEL_9f33729d16874aa2b7e38743e0f96c83",
            "value": "model-00003-of-00004.safetensors: 100%"
          }
        },
        "919f622c4b594e2499103632a8bbe93f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b14aeaa07cb4d8995b24c6692bd6797",
            "max": 3372033380,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0ed8aae20d164d10883a0a90cff22132",
            "value": 3372033380
          }
        },
        "2f9d7a4281124747b4982683cbf5749e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7d5617bf290407aa5c8eec6306ba853",
            "placeholder": "​",
            "style": "IPY_MODEL_af9946bbd3dc4120af0cff7eb23b8795",
            "value": " 3.37G/3.37G [00:10&lt;00:00, 502MB/s]"
          }
        },
        "dfa095324480404db4b2afa2fa2eadd7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1a8ea075a0c7495ca65a6e0c7a04b6d1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f33729d16874aa2b7e38743e0f96c83": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2b14aeaa07cb4d8995b24c6692bd6797": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0ed8aae20d164d10883a0a90cff22132": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f7d5617bf290407aa5c8eec6306ba853": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "af9946bbd3dc4120af0cff7eb23b8795": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "234811e9134c472b9174e680fed95b2d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_da4b0b04334348ba967c2cfce3c5a887",
              "IPY_MODEL_27d8e1dfa108416a978dcc40ddf42b4c",
              "IPY_MODEL_edf7bb666eb04b8ea54b2c4dc50a023c"
            ],
            "layout": "IPY_MODEL_4bf87a423c194fc099efc2d78cd6c3c7"
          }
        },
        "da4b0b04334348ba967c2cfce3c5a887": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b821c746b5d348eda7b3c27219d9a20d",
            "placeholder": "​",
            "style": "IPY_MODEL_c7c6a67636eb486bb23f5224e5c9200a",
            "value": "model-00004-of-00004.safetensors: 100%"
          }
        },
        "27d8e1dfa108416a978dcc40ddf42b4c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b5162183b39446468e33a2cdca30f758",
            "max": 1158267008,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_23b62212dece410dbe2a85c00196da9c",
            "value": 1158267008
          }
        },
        "edf7bb666eb04b8ea54b2c4dc50a023c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ea1299a844ed42e082cea091c8ce195f",
            "placeholder": "​",
            "style": "IPY_MODEL_4115eb4d5dd44bae83056a4e63dd3b10",
            "value": " 1.16G/1.16G [00:01&lt;00:00, 1.24GB/s]"
          }
        },
        "4bf87a423c194fc099efc2d78cd6c3c7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b821c746b5d348eda7b3c27219d9a20d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c7c6a67636eb486bb23f5224e5c9200a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b5162183b39446468e33a2cdca30f758": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "23b62212dece410dbe2a85c00196da9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ea1299a844ed42e082cea091c8ce195f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4115eb4d5dd44bae83056a4e63dd3b10": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "74cb90588496488d9714f10cb77af3b0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0f15dc2ad6404f0d828696a1d150be9e",
              "IPY_MODEL_deb3ae63d3fa486bb9208dd772e4b941",
              "IPY_MODEL_dd9d7e48e0e5412daa6a307f063f6727"
            ],
            "layout": "IPY_MODEL_808bb5cc236744638b91201e7e09aa84"
          }
        },
        "0f15dc2ad6404f0d828696a1d150be9e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_693372c9491d4e149984f5725bacdd9e",
            "placeholder": "​",
            "style": "IPY_MODEL_2866c066ebad4f8e954c40b23031fc62",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "deb3ae63d3fa486bb9208dd772e4b941": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_45df16423e9548dab1793573ff0f4884",
            "max": 4,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3b1cd9915698483ebaf87d4635beaa4d",
            "value": 4
          }
        },
        "dd9d7e48e0e5412daa6a307f063f6727": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ed121ee923374c44bb486db6e5fbf2c9",
            "placeholder": "​",
            "style": "IPY_MODEL_bf2f622bb9cd4c4ea70d33ace46163ed",
            "value": " 4/4 [00:06&lt;00:00,  1.42s/it]"
          }
        },
        "808bb5cc236744638b91201e7e09aa84": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "693372c9491d4e149984f5725bacdd9e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2866c066ebad4f8e954c40b23031fc62": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "45df16423e9548dab1793573ff0f4884": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3b1cd9915698483ebaf87d4635beaa4d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ed121ee923374c44bb486db6e5fbf2c9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bf2f622bb9cd4c4ea70d33ace46163ed": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8296f821dad64002a4dea006b2c6b19c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_aa0344691dce4ab5a174eb3870953e7c",
              "IPY_MODEL_d49597a1e02244819b4061fa87346ca8",
              "IPY_MODEL_e9a3bd725fd547fcbf5eacefc65a975f"
            ],
            "layout": "IPY_MODEL_187ef593f27a428da305efa910b0e191"
          }
        },
        "aa0344691dce4ab5a174eb3870953e7c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_56150486ad3e4004af0ff41742c94e04",
            "placeholder": "​",
            "style": "IPY_MODEL_57f83c3b71ab4a88880a869537678997",
            "value": "generation_config.json: 100%"
          }
        },
        "d49597a1e02244819b4061fa87346ca8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_159849a3cfb74fd7a80977bcb9879dd8",
            "max": 165,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_50acc2e861954abc9554c690cae30b06",
            "value": 165
          }
        },
        "e9a3bd725fd547fcbf5eacefc65a975f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_05f69e6251514ea48eb1693ec3a78430",
            "placeholder": "​",
            "style": "IPY_MODEL_98bfb758b6a147fc93ecf48eed9e0791",
            "value": " 165/165 [00:00&lt;00:00, 21.1kB/s]"
          }
        },
        "187ef593f27a428da305efa910b0e191": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "56150486ad3e4004af0ff41742c94e04": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57f83c3b71ab4a88880a869537678997": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "159849a3cfb74fd7a80977bcb9879dd8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "50acc2e861954abc9554c690cae30b06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "05f69e6251514ea48eb1693ec3a78430": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "98bfb758b6a147fc93ecf48eed9e0791": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb5d3ce91d18489295ce40bfeb647f26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8e74d760ddab4155b63e56068f4a719f",
              "IPY_MODEL_1947bfabde534fa987e79215cf49cc00",
              "IPY_MODEL_4ca0f5ce85864c629d08fe03b8e7656b"
            ],
            "layout": "IPY_MODEL_6e158395111247d392eca45f7bc43a6d"
          }
        },
        "8e74d760ddab4155b63e56068f4a719f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_15613a5f627e4e94b1cc6b5f60e89130",
            "placeholder": "​",
            "style": "IPY_MODEL_632709a7f02641faacf76a980c9f6d86",
            "value": "tokenizer_config.json: "
          }
        },
        "1947bfabde534fa987e79215cf49cc00": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f5244758d8164b71bb2ca15b506f96c3",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a904dcaddbfb4eb3b99acc1ad560e388",
            "value": 1
          }
        },
        "4ca0f5ce85864c629d08fe03b8e7656b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_044eca01c0314107b266903d68920287",
            "placeholder": "​",
            "style": "IPY_MODEL_be10d0853b08422f8d0eb5b835a29901",
            "value": " 22.8k/? [00:00&lt;00:00, 2.65MB/s]"
          }
        },
        "6e158395111247d392eca45f7bc43a6d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "15613a5f627e4e94b1cc6b5f60e89130": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "632709a7f02641faacf76a980c9f6d86": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f5244758d8164b71bb2ca15b506f96c3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "a904dcaddbfb4eb3b99acc1ad560e388": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "044eca01c0314107b266903d68920287": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be10d0853b08422f8d0eb5b835a29901": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bc07f92151a94144bd172d8b0cee312b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_68c7876595f54be79239b9e9d5cc060a",
              "IPY_MODEL_4ef28720654642a6930f1626f3614d7d",
              "IPY_MODEL_d5019b1026624b2db6222dc6c7675445"
            ],
            "layout": "IPY_MODEL_6388045a56d4442ca1ab4b5d3223e864"
          }
        },
        "68c7876595f54be79239b9e9d5cc060a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1bb6b53d018847e28ceec92967305253",
            "placeholder": "​",
            "style": "IPY_MODEL_8a8b670769844150a9347b036bf614c3",
            "value": "tokenizer.json: 100%"
          }
        },
        "4ef28720654642a6930f1626f3614d7d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bd8db8bcc0924f07822c37550b0f1e1e",
            "max": 27868174,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c649e647eb534c1281b57b8e75012f95",
            "value": 27868174
          }
        },
        "d5019b1026624b2db6222dc6c7675445": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_13707c1ee0e24d84962157afb6de134f",
            "placeholder": "​",
            "style": "IPY_MODEL_1ea56ef7280a4854beefd515b1ec0212",
            "value": " 27.9M/27.9M [00:00&lt;00:00, 57.4MB/s]"
          }
        },
        "6388045a56d4442ca1ab4b5d3223e864": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1bb6b53d018847e28ceec92967305253": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8a8b670769844150a9347b036bf614c3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bd8db8bcc0924f07822c37550b0f1e1e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c649e647eb534c1281b57b8e75012f95": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "13707c1ee0e24d84962157afb6de134f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1ea56ef7280a4854beefd515b1ec0212": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d31bc0b1c75b42d384442a966291cafb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_61392f2309ac462e858c8510935c708f",
              "IPY_MODEL_30daf6694d5643688031af64657a5b21",
              "IPY_MODEL_46ec6fd59dbf4546af573356153f06f0"
            ],
            "layout": "IPY_MODEL_014ea07f613b46e5a52887e017e297a0"
          }
        },
        "61392f2309ac462e858c8510935c708f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_14d34b3c3d52493985c3d65246b55009",
            "placeholder": "​",
            "style": "IPY_MODEL_aeee2eeaa46b4f0d9dbec6eb1ba8dcbd",
            "value": "special_tokens_map.json: 100%"
          }
        },
        "30daf6694d5643688031af64657a5b21": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c2d0d41cbf894458b91380f3fd5c187e",
            "max": 446,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0c2f712d7f6b443199c54976d225094f",
            "value": 446
          }
        },
        "46ec6fd59dbf4546af573356153f06f0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_94cc5d592a164b838185609f5d82064e",
            "placeholder": "​",
            "style": "IPY_MODEL_958c3e89aa0142559abd0bdd0991e44e",
            "value": " 446/446 [00:00&lt;00:00, 57.7kB/s]"
          }
        },
        "014ea07f613b46e5a52887e017e297a0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "14d34b3c3d52493985c3d65246b55009": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "aeee2eeaa46b4f0d9dbec6eb1ba8dcbd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c2d0d41cbf894458b91380f3fd5c187e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0c2f712d7f6b443199c54976d225094f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "94cc5d592a164b838185609f5d82064e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "958c3e89aa0142559abd0bdd0991e44e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1e4542c5982a4fa2b6bfbee2787ba728": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_0ace86978f524af3be4a5d3795a70b66",
              "IPY_MODEL_9cfdc7fe2c3d453897eef1e5fcabf530",
              "IPY_MODEL_1e063f1d7ef1420c8d54e73798b46246"
            ],
            "layout": "IPY_MODEL_caa6354355ac4bd8b87106b853a15e1e"
          }
        },
        "0ace86978f524af3be4a5d3795a70b66": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f1f8a6738ba04c6da3c401c83a17b3bd",
            "placeholder": "​",
            "style": "IPY_MODEL_deb19c6d4c414457aa55c72b2f142828",
            "value": "chat_template.jinja: "
          }
        },
        "9cfdc7fe2c3d453897eef1e5fcabf530": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f7d2c370a02e446aa3eefc5b16b11808",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c43418c254e247058506b1df18d64f78",
            "value": 1
          }
        },
        "1e063f1d7ef1420c8d54e73798b46246": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c4c16513140e4132bc5252fb9e7cb6ed",
            "placeholder": "​",
            "style": "IPY_MODEL_da20cad56c664e2595c1f79a35a3bf6b",
            "value": " 15.1k/? [00:00&lt;00:00, 1.78MB/s]"
          }
        },
        "caa6354355ac4bd8b87106b853a15e1e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f1f8a6738ba04c6da3c401c83a17b3bd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "deb19c6d4c414457aa55c72b2f142828": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f7d2c370a02e446aa3eefc5b16b11808": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": "20px"
          }
        },
        "c43418c254e247058506b1df18d64f78": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c4c16513140e4132bc5252fb9e7cb6ed": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "da20cad56c664e2595c1f79a35a3bf6b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "state" : {}
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}